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Application of Risk within Net Present Value 
Calculations for Government Projects 


Abstract 

In January 2004, President Bush announced a new vision for space exploration. This included 
retirement of the current Space Shuttle fleet by 2010 and the development of new set of launch 
vehicles. The President’s vision did not include significant increases in the NASA budget, so 
these development programs need to be cost conscious. Current trade study procedures address 
factors such as performance, reliability, safety, manufacturing, maintainability, operations, and 
costs. It would be desirable, however, to have increased insight into the cost factors behind each 
of the proposed system architectures. This paper reports on a set of component trade studies 
completed on the upper stage engine for the new launch vehicles. Increased insight into 
architecture costs was developed by including a Net Present Value (NPV) method and applying a 
set of associated risks to the base parametric cost data. The use of the NPV method along with 
the risks was found to add fidelity to the trade study and provide additional information to 
support the selection of a more robust design architecture. 


Introduction 

NPV is a technique that is used to assess the viability of projects based on the projected receipts 
and disbursements over the projects planning horizon. It can, however, become difficult to 
arrive at credible single point estimates for some of these projects. Increases in project 
complexity, increases in planning horizons, and the need to engage multiple subcontractors are 
all factors that increase the risk in developing an accurate NPV. One possible approach to 
address this problem is to incorporate the risks associated with each of these factors into the 
process to arrive at a project NPV. 

This paper explores a methodology to address the risk in developing NPVs for complex 
aerospace projects. First, a test project, the need for and the current development status of a J-2X 
to support the new ARES I and ARES V launch systems is discussed. Next, two approaches to 
modifying NPV, those by Davis [1] and Riddlehoover [6], are presented. Finally, a modified 
approach based on the work by Davis and Riddlehoover is presented and applied to the J-2X 
engine nozzle trade study. The paper concludes with a summary of the conclusions drawn from 
this effort and the identification of further work should be pursued. 
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Background 


The 2004 announcement from the President to retire the Space Shuttle fleet also included the 
development of a new class of launch vehicles to maintain the U.S. presence in space. The first 
to be developed is the Crew Launch Vehicle (CLV), known as ARES I. This vehicle uses an 
inline configuration architecture with a Reusable Solid Rocket Motor (RSRM) as the booster 
propulsion system. This motor will propel the vehicle into the low atmosphere where an 
additional upper stage engine will then place the vehicle into low earth orbit. The engine selected 
for the upper stage propulsion is a modified J-2, known as J-2X. The crew capsule will sit atop 
the entire vehicle. •! 

An additional launch vehicle is also being developed to carry supplies and large payloads 
required for the International Space Station (ISS) and lunar missions. This vehicle, ARES V, has 
a significantly larger architecture than the ARES I due to the requirement to carry the larger 
payloads. ARES V is configured with two Reusable Solid Rocket Motors (RSRM) and five RS- 
68 engines as the core stage booster engines. The propellants for the RS-68 engines are also 
stored in a large external tank. This vehicle also requires an upper stage propulsion system and a 
similar J-2X engine was selected for this stage as well. The ARES V upper stage engine will 
require slightly different performance than! the ARES I upper stage engine. 

The J-2X (the modified J-2) engines for the upper stages on both vehicles will be based on 
heritage Apollo hardware, but modified to meet performance and safety requirements (man 
rating). The propellants selected for the J-2X engine are liquid hydrogen (LH2) and liquid 
oxygen (LOX) for their high performance characteristics. Due to the differences in the thrust 
requirements of both vehicles, the engine is being designed for the larger thrust since it can be 
throttled back to a lower thrust. There are several requirements given to the engine based upon 
mission and vehicle requirements for thrust, specific impulse, weight, maintenance, and volume. 
Each of these requirements is driven down to the subsystems and components of the engine. 

There are four major subsystems of the engine including: turbomachinery, combustion devices, 
avionics and instrumentation, and valves, lines and ducts. For combustion devices, there are five 
major components. These components are the main injector, main combustion chamber, gas 
generator, regeneratively-cooled nozzle, and nozzle extension. The focus of this paper will 
discuss the trade studies being developed for the regeneratively-cooled nozzle. The project 
drilldown from mission requirements through selection of the nozzle for this project is shown in 
Figure 1. 

Preliminary trade studies have looked at alternative nozzle designs to select the appropriate 
conceptual architecture to meet mission, vehicle and engine system requirements. There are 
several decisions that have already been made regarding the nozzle from the engine systems 
team and lessons learned from past liquid rocket engines. Some of these decisions include 
overall envelope, contour configuration, and cooling requirements. The vehicle had previously 
designated the overall envelope in which in the engine will be present. Due to standards in liquid 
engine design, this overall engine envelop^ defines the general envelope of the nozzle. 
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Figure 1. Project Drilldown to the J2 Upper Stage Engine. 

The purpose of this trade study was to down select between a channel wall nozzle (CWN) and a 
tube wall nozzle (TWN), of which there are two configurations for each. The major decisions to 
be made were the configuration of the coolant pass scheme and manifold locations. There are 
several types of cooling schemes that could have been selected for each configuration of nozzles. 
A one-pass coolant scheme is a single pass from the aft end of the nozzle to the forward end of 
the nozzle. The second type of cooling scheme is a 2-pass. This is where the coolant inlet is 
toward the forward end of the nozzle, makes a downpass, turns around, and does an up-pass to 
collect in the forward end of the nozzle. Another type of coolant scheme is a 1 .X coolant pass. 
This is where the inlet is located somewhere between the forward end and aft end of the nozzle. 
The coolant makes a downpass, turns around and makes an up-pass and collects in the forward 
end. The X designates the location on the nozzle. For example, if the inlet manifold were located 
half-way down the nozzle, it would be designated as a 1.5 coolant pass. 

This study had significant emphasis on the costs for each architecture. There are four nozzle 
configurations that were evaluated. The 1 .6 Pass TWN was selected as the baseline based on a 
team decision for performance and application of new technologies. Option 2 is the 2.0 Pass 
TWN. Option 3 is the 1 .0 Pass CWN and Option 4 is the 2.0 Pass CWN. Each of these options 
has unique technical and programmatic differences, thus affecting the cost structure for each. 
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Problem Statement 

It is difficult for organizations to evaluate costs for large complex aerospace projects due to the 
high technological and programmatic risks. The difficulties will increase when the project has 
an extended planning horizon. The risks that lie in future years add to the difficulties in making a 
decision of which architecture to pursue in the present. A common method for firms to evaluate a 
project’s cost is using Net Present Value (NPV). Salvatore [10] describes that NPV is “equal to 
the present value of the expected stream of net cash flows from the project, discounted at the 
firm’s cost of capital, minus the initial cost of the project.” This evaluates all the positive and 
negative cash flows and brings them all to a common period of time for all options. Using the 
NPV method a firm can include their cost of capital, or minimum attractive rate of return 
(MARR), which they require to make the project worth pursuing. If the NPV of the project is 
greater than zero, the firm could invest knowing that the MARR is met and it should not be 
losing money by investing in the project. This method also works when evaluating multiple 
mutually exclusive projects. If a firm were to evaluate multiple projects and calculate a NPV for 
each, the best project would be that of the highest NPV. This method is relatively simple and 
straightforward which aids in explaining the analysis to senior management. There are multiple 
sources that illustrate applications of NPV, such as Park [8], Blank and Tarquin [1] and 
Eschenbach [3.] 

Large NASA projects can be a bit more challenging since the government does not operate using 
profitability. Public sector projects still require financial analysis to determine how capital will 
be allocated and which option(s) best benefits the agency and society. Methods such as cost- 
benefit analysis and modified cost-benefit analysis have been developed to evaluate such 
government projects. These projects can also be evaluated using the traditional techniques such 
as NPV. The methods for calculating NPV for public and private sector projects are similar; 
variations occur because of the nature of public sector projects. The primary differences lie 
within the final evaluation of the final NPVs. As often seen in government contracts the lowest 
bidder or lowest cost project will receive the funding to complete the proposed work. The 
government will then typically select the lowest NPV to invest in as opposed to the largest NPV 
for corporate firms. 

There are some potential problems with using NPV to evaluate projects. NPV is strictly a 
financial tool. There are often other factors such as technical, political, environment, or 
programmatic that are not included in the NPV calculation. An entity, whether it is the 
government or a corporate firm, must carefully consider all factors and their associated risks 
when evaluating a project or a set of options. This is of particular importance when the project 
spans over several years or decades. The further out in future years that a project is evaluated the 
higher the risks of that project. The evaluating entity cannot predict what events may happen in 
future years that could influence the initial calculated costs of the project or options. External 
risk factors can influence the financial data and subsequently the NPV for evaluating options. 

Previous Work 

Several authors have researched the topic of applying risk in financial evaluations of a project. 
Davis [2] presented a method called “Net Present Value Risk- Adjusted (NPVR)”. The technique 
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was used to address critical risk factors within the existing NPV model, and reported that NPVR 
exposed risks within critical areas that allows for better predictions than the original NPV model. 
Davis develops a specific model that evaluates six key areas of new product development: value 
chain, market segment, innovation, capabilities, and interaction and specification assessments. 
Each of these areas is rated on a scale of 1 - 5, with 5 being a high chance of success and 1 being 
a low chance of success. A weighting system is then applied to each of the key areas. The risk 
weighting includes market, technical and user. The calculation is shown in Equation 1 . 


NPVR = ^ + + + + + ^ x NPV 

10 

where, 

NPVR = Net Present Value Risk- Adjusted 

a, b, c, d, e, f = Value chain, market segment, innovation, capabilities, and 
interaction and specification assessments, respectively 
M = Market Risk 
T - Technical Risk 
U= User Risk 
NPV = Net Present Value 


( 1 ) 


It can be seen from the model above that risk is applied to the NPV calculation using a subjective 
expert rating system and applying weights to each of the risk factors. This model demonstrated 
that the application of risk to the NPV calculation helps illustrate how risks may affect the 
success or failure of a project by highlighting variability in the NPV results. 

Riddlehoover [9] proposed a similar approach. The author looked at several locations to open a 
new facility. In this study, Monte Carlo simulations were run on the new facility locations using 
the following factors: initial investment, annual cost, annual benefit, and interest rates. Annual 
Worth (AW) was used in the study rather than NPV. The author found that there was an 
observed difference between the annual worth for each of the projects, however it did not 
accurately reflect the risks for each of the projects. The author further developed his annual 
worth model using a Certainty Equivalent Value (CEV) model which included a risk factor for 
the project selection. The inputs into the CEV model included three factors: Labor, 
Transportation, and Real Estate. For input into the model, Riddlehoover applied a weighting 
factor to each of the location factors similar to what Davis did. He then applied a risk factor for 
each of the final locations being considered based on an expert rating system. This risk factor 
presented in the paper used a decimal form between [0, 1], however it was discussed that a scale 
of 1 - 5 could be used to achieve similar results. The weighting factor and risk factor are then 
rolled into a combined risk factor (single number) for each of the options. This combined risk 
factor in conjunction with the expected value and standard deviation is used in the CEV model as 
shown in equation 2. 

CEV = E\x\- R x <y[x] (2) 

where, 

CEV = Certainty Equivalent Value (Adjusted Annual Worth in this article) 

E[x] - Expected value from Monte Carlo simulation 

R = Combined Risk factor 
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c[x] = Standard Deviation from Monte Carlo simulation 

In his work Riddlehoover found that the highest Annual Worth (AW) was now down the list for 
project selection based on a high risk. This model was very simple, but the model had a much 
higher fidelity compared to the previous Monte Carlo simulations of AW. Higher fidelity means 
that there were distinctive differences as compared with the previous AW calculations. 

Both the CEV and NPVR models applied a factor weighting system to each of the risk factors 
being considered in the model. This data was then applied in conjunction with the existing NPV 
data to obtain a new value, reflecting financial and risk data. Each author saw interesting results 
and in some cases a reversal of the initial decision for project investment. It was shown that 
sometimes a project with the strongest financial benefit according to NPV calculations is not 
always the best choice when corresponding risks are applied. These models were applied to both 
general and specific situations, but certainly show promise for new applications such as 
government project evaluation. 

Method and Scope of Work 

This paper applies a NPV-Risk method to the trade study being completed on the J-2X 
regeneratively-cooled nozzle. This NPV-Risk method is derived from the previous work of 
Davis and Riddlehoover. The method being used will include a Monte Carlo simulation of the 
costing data for a selected time period. The input data for this Monte Carlo simulation will be 
based on required labor for component design and production, required materials, and 
appropriate rate increases for the time period. Using the NPV and statistical data derived from 
the Monte Carlo simulation, a NPV-Risk or CEV model will be developed to adjust each of the 
options. This adjustment will include the application of risks for the nozzle options being 
considered. There will be some differences discussed in the CEV model for specific application 
to government projects. 

Initial Cost Estimating 

In order to develop the required input data into the CEV or NPV-Risk model, it was decided to 
run a Monte Carlo simulation to obtain these values. The primary output that is required for the 
CEV model is an expected value and standard deviation for the net present value. A NPV could 
be calculated for each of the options, but this would not supply an appropriate standard 
deviation. A calculated NPV does not provide any information about the range of values that are 
input into the model. Each of these values could have a possible affect on the final outcome. 
There were five variables of interest for the simulation: Government Labor Rate Increase (%), 
Government Inflation-Interest Rate (%), Contractor Labor Rate Increase (%), Material Cost 
Increases (%), and Contractor MARR Value (%). 

The first assumption for estimating the associated costs was the time period to study. A period of 
15 years was selected in one year increments. This time period will cover the major milestones 
and a few years beyond to allow for the manufacturing to stabilize. Component testing and full 
engine testing is scheduled to begin in 2010. A completed nozzle will be required at this time 
with design and manufacturing leading up to this point. The first ARES I flight is scheduled for 
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2013 with a major increase in production and flight support leading to this peaking. Another 
major milestone is 2018 in which the lunar mission will begin. A time period of 3 years beyond 
this was used for the analysis. 

The cost model is broken down into the government side and the contractor side for 
completeness within the model. To develop the government costing, the activities were broken 
down into categories. These categories included: trade study development, support from all 
engineering disciplines (computational fluid dynamics, structural, thermal, and design), design 
reviews, sub-scale and full scale testing, general insight, manufacturing support, and flight 
support. Each group was calculated per the year based on the number of required Full-time 
equivalents (FTE’s) to support that task for that year. 

After the FTE’s were developed for each year, a rate needed to be applied to put these hours into 
dollar equivalents. For years 2006 through 2010, there were rates previously developed by a 
NASA business office. In order to develop rates for years beyond 2010, the previous percentage 
increase in years was averaged and applied to the future years. A Monte Carlo simulation was 
applied to the NASA FTE labor rates to further understand how these affect the present and 
future values within the cost development. 

An interest rate was selected to be applied to the labor dollars for the government work since a 
MARR value is not applicable. This is on average the annual cost of living increase per year 
(inflation rate). Since the government does not need to make a profit, I feel that this rate is 
appropriate to apply in the analysis. The total cost for each year was calculated and the net 
present value of the entire government support was determined with the appropriate rates 
applied. 

To determine which system architecture is considered optimum, the manufacturing and 
contractor design costs must be understood. This data is developed based on estimated costs 
from the prime contractors costing department. These costs are fully developed from a historical 
experience base of hours for design, machining, and fabrication based on previous engines. 
Material and tooling costs are estimated based on vendor pricing and historical records. It is 
assumed that the costs will be spread out over multiple years during first article manufacturing. 
For the manufacturing costs for each of the nozzles, it is assumed that the materials will be long 
lead items and must be purchased 1 -2 years in advance. It is common to stock pile certain raw 
materials to minimize foundry operations. The touch labor and fabrication time is also assumed 
to be multiple years. It is assumed that the touch labor, materials, and manufacturing engineering 
support will be the only reoccurring costs to produce nozzles. Design and tooling will be the only 
non-recurring costs associated with the first article and regular manufacturing activities. 

The reoccurring costs expanded over the entire time period is based on the number of nozzles 
required to support missions for the year. The ongoing support activities are generally assumed 
to be the same regardless of the nozzle selected and the number manufactured. This is offset to 
year 2009 since this is when testing starts and design activities should be wrapping up. This 
number is based on the mission requirements to support space station and lunar flights. There 
were several rates applied to the contractor’s costs to account for increases in materials and 
labor. The first rate applied to the contractor costs was a percent increase in labor rates. It is 
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assumed that the labor rate increase is less than the government rates since the contractor does 
not have as much overhead as the government. This assumption is that several of the facilities 
and machinery being used are government furnished. The second rate applied was a percent 
increase in material costs. Since several of the materials are exotic and purchased in lower 
quantities, it is difficult to get vendor’s support in supplying materials. These rates were only 
applied from 2010 out since several of the long lead items are being ordered in the near term and 
labor costs are generally known for the next 4 years. 

The Minimum Attractive Rate of Return (MARR) applied to the analysis is assumed to be fairly 
low since there is generally not a high profit margin in the aerospace sector. There are already 
some markups associated with the material and manufacturing costs given by the vendor. The 
assumed MARR value is based on discussions with the NASA business office. 

As previously discussed, five factors were applied to the Monte Carlo Simulation: Government 
Labor Rate Increase (%), Government Inflation-Interest Rate (%), Contractor Labor Rate 
Increase (%), Material Cost Increases (%), and Contractor MARR Value (%). The following is 
rationale for why each of the minimum and maximum values were chosen for the simulation: 

. Government Labor Rate Increase - The baseline was 3.76% based on an average increase 
in labor rates through 2010. These rates were estimated by the authors based on experience. 
The minimum increase per year was 2.0% and the maximum was 6.9%. 

. Inflation (Interest) Rate Applied - The average inflation rate was 2.30%. This was based 
the average increase of living increases applied to government salaries on a yearly basis. 
The maximum increase based on historical data was determined to be 3.83% and the 
minimum rate was 1.55%. These are based on rates from 1996 through 2006 (Financial 
Trend Forecaster, 2007). 

• Rate Increase for Contractor - The rate increases for the contractor are generally less than 
the government due to less uncertainty in overhead. The baseline rate used was 2.5%. A 
minimum value chosen for the analysis was 0.9% and a maximum value was 3.7%. This is 
at the lower range for the government rates. These rate increases are often tied to inflation 
since the profit is built into the contractor MARR value. However, the contractor can still 
request rate increases yearly. 

• Material Cost Increases Per Year — The base rate chosen for the analysis was 5.5%. A 
minimum value for the simulation of 4% was selected. A maximum value of 11% was 
chosen. It is expected that the material costs could be higher than 5.5% rather than lower 
because of the exotic and low quantity of materials required. This data is based upon casual 
conversations with a vendor. 

• Contractor MARR Value - The contractor has already included some of their overhead in 
the rates applied to support and manufacturing, so a lower MARR value was chosen as a 
baseline (12%). To run the simulation, a lower value of 10% was selected and 20% 
selected as a high value since this is more in line with other industries. 

A Monte Carlo simulation was completed using @Risk Monte Carlo software package from The 
Palisade Corporation [7]. The input variables were discussed above and the output variable was 
NPV. The distribution for each of the input variables was defined as a triangular distribution. 
This type of distribution was defined since minimum, most likely, and maximum values could 
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easily be defined for each of the variables. This type of rough simulation method is used for 
input variables that are lacking a fully defined dataset. The triangularly distributed input 
variables have been used commonly in other Monte Carlo simulations [5]. For completeness 5 
simulations were completed for each option and 1,000 iterations were completed for each 
simulation. A different random generator seed value was selected for each simulation completed. 
It was observed that each simulation for each option demonstrated convergence at less than 1.5% 
level. For each option, the value obtained from the simulations was averaged to obtain one NPV. 
These values can be seen in Table 1. All dollar values are shown in thousands ($lk). 

Table 1 . NPV Obtained for Each Trade Study Alternative. 


Alternative 


NPV 

Option 1 -1.6TWN 

$ 

226,162 

Option 2- 2.0 TWN 

$ 

228,048 

Option 3-1.0 CWN 

$ 

240,379 

Option 4- 1.9 CWN 

$ 

239,743 


Since this decision is being made from the government side, it would be based on the lowest net 
present value of the options. Based on the NPV, the government would select option 1, since this 
has the lowest NPV of all four. This only considers cost for the trade studies though. It can be 
seen that there is not a significant difference in costs between the four options based on relative 
differences. The maximum difference between the highest and lowest NPV’s is 5.9%. This 
analysis did not elect to include learning curves since experience with similar development 
projects has tended to make the researchers concerned about the accuracy of the learning curve 
results. The government will pay a fixed amount for a certain number of nozzles per year, so the 
learning curve is not actually applicable in this situation. Also, there is an aging workforce at the 
contractor that are eligible to retire in the next several years, so any learning curve may go back 
to square one with a new workforce. Due to these reasons the learning curve was excluded, but it 
may be an area for further investigation. 

Changes to Applied Model 

The Monte Carlo simulation did not show significant relative differences between each of the 
options. Since this is a long term project and the wrong option selection could have long term 
effects, it was decided to further develop the cost model. Based on previous work, it has been 
shown that risk factors can be applied to the net present value in an attempt to bring out more 
fidelity in the cost data. 

There are four primary steps as presented by Riddlehoover to apply the CEV Model. The first 
step is to develop appropriate weights for each of the factors selected for the risk model. These 
weights should be normalized and in decimal format. The next step is to develop and apply a 
rating system to each of the risk factors for each of the options. This will provide a matrix of 
values for each factor an option. The third step is to develop an overall risk factor. Riddlehoover 
presented a summation method to develop a combined risk factor for each option. The modified 
method for this trade study can be seen in Equation 3 . 
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where, 

W = Decimal Equivalent of Factor Weight 
R = Risk Factor, based on scaling 1-5 

i = Factor (Government Labor Rate Increase, Government Inflation-Interest Rate, 

Contractor Labor Rate Increase, Material Cost Increases, and Contractor MARR 
Value) 

k = Total Number of Factors 

Once the combined risk factors are calculated for each of the options, the CEV can be 
determined for each of the options and compared. This CEV value now includes the application 
of risk applied to the model based on the selected risk factors. The CEV required a slight 
modification since the previous literature only covered commercial firm projects. Since 
commercial firm projects are generally for profit, when risk is applied to the model it creates a 
lower value as seen in Equation 2. If the project has a higher risk, the NPV is then lowered and 
would not be as attractive to the firm. However, government projects select an option based upon 
the lowest NPV, so the combined risk factor and standard deviation must be added. This 
nomenclature properly applies the risk to the model. Now, if one project has more risk, the NPV 
will be higher and NASA would not be as likely to select that project. The modified CEV can be 
seen in Equation 4. 

CEV = E[x\ + Rx ct[x] (4) 

where, 

CEV = Certainty Equivalent Value (Adjusted NPV) 

E[x] = Expected value from Monte Carlo simulation 
R = Combined Risk factor 

a[x] = Standard Deviation from Monte Carlo simulation 

Application of Model to Nozzle Trade Study 

Before any steps are completed for the applied CEV model, the appropriate factors to study must 
be decided upon. It was shown in the previous models that different variables than the NPV 
variables could be applied within the risk model. However, it was decided that the most 
appropriate values to include in this model are the same factors as used to run the Monte Carlo 
simulation. These factors were Government Labor Rate Increase, Government Inflation-Interest 
Rate, Contractor Labor Rate Increase, Material Cost Increases, and Contractor MARR value. 
Now, the methodical steps can be completed as discussed in the previous section. 

The first step in the model is to develop the factor weights. The factor weights were based on 
expert rating. The higher values equate to a higher risk for that particular factor. Subsequently, a 
lower value equates to a lower risk for that factor. The factor weights can be seen in Table 2. 



Table 2. Factor Weights for the Trade Study. 


Factor 

Weight 

Government Labor Rates 

0.05 

Inflation (Interest) Rates 

0.1 

Rate Increase for Contractor 

0.35 

Material Cost Increase 

0.45 

Contractor MARR Value 

0.05 


It can be seen from the table that the values range from lower risk (0.05) to a higher risk (0.45). 
The government labor rates were weighted low because these are generally tied to the inflation 
rates. These rates have not seen any drastic increases over the last few decades are fairly 
predictable. The overhead is generally constant for these and fluctuations are stable and within 
expected values. This value was rated at 0.05 because of these reasons. The next factor, Inflation 
(Interest) Rates, was weighted fairly low as well. Plenty of historical data is available for 
inflation rates broken down by years and even months. This data does not fluctuate significantly, 
although there are some increases that are higher than others. Because of these reasons, this was 
ratedO.10. 

The rate increase for the contractor was rated higher amongst the factors. The contractor can 
increase rates in the future years for no apparent reason. These clauses are within the contract to 
give the contractor authority to do this. If the contractor has foresight that profitability is not to 
the level required of management, they may request a rate increase. Although these increases are 
spelled out in the contract, they are not always accounted for in the NASA budget. This was 
rated at 0.35. The contractor may not increase the rates drastically in one year, but then the 
following year may raise it. Although the rates are low, it is considered a risk since several of the 
costs are developed by the contractor. 

Another high risk factor is the material cost increase which was rated at 0.45. Material costs are 
outside of the government’s control and it was determined to be a serious threat to the project. 
Since materials are purchased in relatively small quantities as compared with other industries, no 
economies of scale or political power can be asserted. Also, several of the exotic aerospace 
materials are being purchased by multiple customers, so the future costs of the material can be 
unpredictable. This was determined to be the highest risk amongst all the factors. 

The final risk weighting is the MARR value. MARR values are generally controlled by 
contractual means within the government. Thus, a contractor is limited on the amount of profit it 
can make off of certain projects. The government understands that the company must make a 
profit, so once all costs are figured, a MARR is applied. These values generally do not fluctuate 
much, so it was determined that a low risk value of 0.05 was appropriate. 

The next step for application of the model is to determine a value for each of the risk factors for 
each of the options. A matrix was developed to aid in the assessment. A scale from 1 to 5 was 
used with 5 having the highest risk and 1 having the least risk associated. The descriptions for 
each of the rating scales and factors can be found in Table 3. 
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Table 3. Sealing Factors and Descriptions for Each Factor 


Risk 

Level 

Government 

Labor 

Rates 

Inflation 

Rates 

Rate Increase for 
Contractor 

Material Cost 
Increase 

Contractor 

MARR 

1 

NASA has little 
involvement in 
project other than 
procurement 
activities. This is 
an iteration of a 
previous design. 

Economy is stable. 
The dollar is highly 
valued in foreign 
markets, 

unemployment is 
extremely low and 
Employment Cost 
Index is on the rise. 

Contractor does not 
increase rates. The 
product is somewhat 
profitable and there 
is no need for 
additional rate 
increases. 

Material costs have 
decreased; Increase 
supply in the market 
allows prices to drop. 
Lead times are only 
a few weeks. Special 
orders are 
accommodated. 

Contractor MARR 
value is significantly 
lower compared to 
other NASA projects. 

2 

NASA has minimal 
involvement in the 
project, limited to 
design reviews, 
Technical 
Interchange 
Meetings, and risk 
closures. 

Intermediate Level 

Contractor increases 
rates slightly to 
adjust for minor 
project risks. This 
rate increase should 
not affect NASA. 

Intermediate Level 

Intermediate Level 

3 

NASA has daily to 
weekly involvement 
in project, including 
data reviews and 
discussions with 
contractor to close 
issues. 

Inflation has 
increased slightly but 
not at significant risk 
of fluctuations. 
Fluctuations have 
averaged out to be 
at a low to medium 
level. 

Rates have increase 
due to unforeseen 
risks in design or 
production. Rate 
increase is agreed 
upon by both parties 
without any major 
issues. 

Material costs have 
remained fairly 
stable. There may be 
a slight increase, but 
not greater than 
expected to adjust 
for inflation. 

Contractor MARR is 
of an average value 
when compared with 
other NASA projects. 

4 

NASA does 
reviews of data, 
including 
independent 
modeling of 
selected analyses. 
Verification plan 
reviews and 
concurrence of all 
closure rationale. 

Intermediate Level 

Rates have 
increased due to 
fairly significant 
risks. This rate 
increase is slightly 
higher than 
anticipated by 
NASA. 

Materials have 
increased 
significantly. Some 
materials are difficult 
to obtain, but most 
can be obtained for a 
premium price. Lead 
times are in excess 
of 6-12 months. 

Intermediate Level 

5 

NASA performs 
complete and 
independent 
analyses of each 
task. Review of 
verification plan 
and witness to 
certification testing. 

Significant increase 
in prices of 
consumer goods. 
The dollar is 
unstable and 
purchasing power 
has decreased 
significantly. 

Significant contractor 
rates increases to 
support profitability. 

Product costs in 
design or production 
have increased. 
NASA does not 
support these rate 
increases. 

Materials are 
extremely difficult to 
purchase and have 
increased 

significantly in cost. 
Lead times are in 
excess of 24 months 
and some alloys are 
rare. 

Contractor MARR is 
significantly higher 
than most other 
NASA projects. 


This methodical rating approach based on the matrix in Table 3 was used for each of the options. 
For Option 1, the values can be seen in the table below for the five risk factors. The Government 
Labor rates were scored at a 2 since NASA would have some involvement in the development of 
this design. A similar design of this configuration has been completed before, so some review of 
the concept would be required, but not significant modeling or analysis. The interchanges with 
the contractor would be limited to design reviews and technical interchange meetings. There 
would not be a need to hold daily or weekly meetings to review the design with NASA and the 
contractor. 

Inflation rates were scored at 1 due to the fairly stable economy. It can be seen that Inflation 
Rates did not change for any of the options above and they are all on a level playing field for this 
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risk factor. This factor could potentially be eliminated since all values were the same, but was 
kept in the model for comparisons. The dollar is still holding its value and there have not been 
any significant increases or decreases in inflation/deflation in the past several months. The third 
factor, Rate Increase for the Contactor, was scored at 4. It is assumed that the new changes to the 
configuration may not be entirely understood when the detailed design is completed and in order 
to maintain profitability, the contractor would increase the labor rates. This increase may be 
higher than anticipated by NASA, thus a higher risk is applied. Option 1 and Option 2 are both of 
TWN configurations, however since Option 1 is applying some new technologies some of the 
profit could be decreased if any failures were to occur. Thus when compared with option 1, it is 
rated slightly higher than option 2 for contractor rate increase. The materials for Option 1 , 3 and 
4 are more exotic and the team may experience some procurement difficulties. The lead times 
could potentially be in excess of 6 - 12 months since they may be difficult to obtain. Due to the 
potential difficulties in acquiring the materials, Option 1 was scored at 4. 

The MARR value for Option 1 was scored at 2. This MARR value is not as low as other projects 
and somewhat average when compared to other NASA projects. Due to this reason, I ranked it at 
an intermediate value. It can be seen that most of the values for all the altematives/options are 
very close. Option 4 is scored slightly higher since this is the furthest from the current experience 
of nozzles and potentially carries the most risk and deserves higher profitability if successful. 
The results of the risk assessments can be found in Table 4. 


Table 4. Risk Factors and Ratings for Each Option. 


Alternatives 



Option 1 

Option 2 

Option 3 

Option 4 

L. 

Government Labor Rates 

2 

2 

3 

4 

W 

o 

Inflation (Interest) Rates 

1 

1 

1 

1 

(0 

LL 

Rate Increase for Contractor 

4 

3 

4 

5 

V) 

Material Cost Increase 

4 

3 

4 

4 

ft 

Contractor MARR Value 

2 

2 

2 

3 


A combined risk factor must be calculated for each of the options. This is completed using 
Equation 3. An example is shown below for Option 1. This calculation was completed for each 
of the options and the values are provided in Table 5. 

£ W t R, = 0 . 05 ( 1 ) + 0 . 10 ( 1 ) + 0 . 35 ( 3 ) + 0 . 45 ( 4 ) + 0 . 05 ( 2 ) = 3.5 

M 

where, 

W = Decimal Equivalent of Factor Weight 
R = Risk Factor, based on scaling 1 - 5 

i = Factor (Government Labor Rate Increase, Government Inflation-Interest Rate, 

Contractor Labor Rate Increase, Material Cost Increases, and Contractor MARR 
Value) 

k = Total Number of Factors 
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Table 5. Combined Risk Factors for Each Option. 


Option 1 

3.5 

Option 2 

2.7 

Option 3 

3.6 

Option 4 

4.0 


Results 

The expected NPV and also the standard deviation were obtained from the original Monte Carlo 
simulation. The values for each of the options can be found in Table 6 and is identical to the 
original NPV shown in Table 2. 


Table 6. Expected Values and Standard Deviation for Each Alternative. 


Alternative 


NPV 

St. Dev. 

Option 1- 1.6 TWN 

$ 

226,162 

$ 

24,331 

Option 2 - 2.0 TWN 

$ 

228,048 

$ 

23,812 

Option 3 -1.0 OWN 

$ 

240,379 

$ 

24,275 

Option 4-1.9 OWN 

$ 

239,743 

$ 

25,628 


Each of these values was used to calculate a CEV for each of the options. The CEV was 
calculated using the modified CEV in Equation 4. An example calculation for Option 1 is shown 
below: 


CEV = £[*]+ R x a[x\ = $226,162 + 3.5 x $24,33 1 = $31 1,321 

where, 

CEV = Certainty Equivalent Value (Adjusted NPV) 

E[x] - Expected value from Monte Carlo simulation 
R = Combined Risk factor 

g[x] — Standard Deviation from Monte Carlo simulation 

Using Equation 4, a CEV value was calculated for each option. These values can be seen in 
Table 7. 


Table 7. Calculated CEV for each Alternative. 


Alternative 

CEV 

Option 1 - 1.6 TWN 

$ 311,321 

Option 2 -2.0 TWN 

$ 292,340 

Option 3-1.0 OWN 

$ 326,555 

Option 4 -1.9 OWN 

$ 342,254 


With the application of risk to the NPV calculation, it can be seen from Table 7 that there are 
distinct differences between each of the options and these are different from the value originally 
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calculated and shown in Table 1. There would actually be a reversal in the decision with the 
application of risk to the model. The government would select Option 2 since this has the lowest 
NPV including the application of risk. Option 1, which was originally selected before application 
of the model, has been moved to the second selection. There is actually a reversal if a selection 
had to be made between the two CWN options. Option 4, which had a lower NPV than Option 3 
in the original calculations, is now higher than Option 3 with the application of risk. 

It is worth noting that in both analyses, NPV and CEV, Options 1 & 2 appear to be preferential 
to Options 3 & 4. Although the rank ordering of the options has some reversal when switching 
analysis methods, these still allow for the decision maker to down select the list of alternatives 
from 4 options to 2, thus a smaller solution space. 


Conclusions and Future Research 

The methodology applied is consistent with previous work applying risks to financial 
calculations such as net present value. It was shown that minor adjustments can be made to apply 
a commercial firm application to a government project. A CEV model was developed that 
considered that the government selects projects based on the lowest NPV as opposed to the 
highest NPV for commercial firms. The CEV model considers risk within the financial 
calculation so a more-informed decision can be made considering more factors than the typical 
NPV financial model. Since the risks applied to the model are replicated throughout the 
calculation, the CEV model was observed to have higher fidelity than a typical NPV calculation. 

The application of risk to the NPV financial calculation was a simple method that provided more 
insight to decision makers. This calculation was applied to a specific trade study for 
regeneratively-cooled nozzles, but could be applied to any long term project. It has been shown 
that the application of risks within NPV calculations can be used for both commercial and 
government projects using slight modifications. The model allows for a balance of the agency 
risk factors and perceived project risks combined into one model. 

This effort noted several areas that could require further research. The first is the impact of using 
alternative techniques to develop the weighting for both the risk weighting and the risk factors. A 
different method aside from the 1 - 5 scale could be used to complete these matrices. Another 
area of further research would be the application of risk factors different from the one used in 
initial NPV calculations. There is further work that can be completed on the Monte Carlo 
simulation as well. The CEV can be incorporated within the Monte Carlo simulation to allow the 
decision maker to quantify the statistical error associated with the cost estimates. The triangular 
distribution was used for simplicity and the input variables selected. If different variables were 
selected other than those used for the original NPV calculation, different distributions would 
need to be investigated. 

Decisions between alternatives cannot be made based solely on financial criteria, even if that 
model incorporates risk. An aggregate decision model must also include non-financial criteria, 
which are often subjective. The next step in more fully evaluating the selected alternatives 
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would be the development of a decision model that incorporates multiple decision criteria and 
weights the criteria based on relative importance. 
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